In the current person Re-identification (ReID) methods, most domain generalization works focus on dealing with style differences between domains while largely ignoring unpredictable camera view change, which we identify as another major factor leading to a poor generalization of ReID methods. To tackle the viewpoint change, this work proposes to use a 3D dense pose estimation model and a texture mapping module to map the pedestrian images to canonical view images. Due to the imperfection of the texture mapping module, the canonical view images may lose the discriminative detail clues from the original images, and thus directly using them for ReID will inevitably result in poor performance. To handle this issue, we propose to fuse the original image and canonical view image via a transformer-based module. The key insight of this design is that the cross-attention mechanism in the transformer could be an ideal solution to align the discriminative texture clues from the original image with the canonical view image, which could compensate for the low-quality texture information of the canonical view image. Through extensive experiments, we show that our method can lead to superior performance over the existing approaches in various evaluation settings.
translated by 谷歌翻译
The activity of the grid cell population in the medial entorhinal cortex (MEC) of the mammalian brain forms a vector representation of the self-position of the animal. Recurrent neural networks have been proposed to explain the properties of the grid cells by updating the neural activity vector based on the velocity input of the animal. In doing so, the grid cell system effectively performs path integration. In this paper, we investigate the algebraic, geometric, and topological properties of grid cells using recurrent network models. Algebraically, we study the Lie group and Lie algebra of the recurrent transformation as a representation of self-motion. Geometrically, we study the conformal isometry of the Lie group representation where the local displacement of the activity vector in the neural space is proportional to the local displacement of the agent in the 2D physical space. Topologically, the compact abelian Lie group representation automatically leads to the torus topology commonly assumed and observed in neuroscience. We then focus on a simple non-linear recurrent model that underlies the continuous attractor neural networks of grid cells. Our numerical experiments show that conformal isometry leads to hexagon periodic patterns in the grid cell responses and our model is capable of accurate path integration. Code is available at \url{https://github.com/DehongXu/grid-cell-rnn}.
translated by 谷歌翻译
立体类像素细分旨在通过左右视图将离散的像素分组为感知区域,以更加协作和高效地分组。现有的Superpixel分割算法主要利用颜色和空间特征作为输入,这可能会对空间信息施加强大的约束,同时利用立体声图像对的差异信息。为了减轻此问题,我们提出了一种立体声超级像素细分方法,并在本工作中具有空间信息的脱钩机制。为了解除立体视差信息和空间信息,在融合立体声图像对的特征之前,暂时删除空间信息,并提出了脱钩的立体声融合模块(DSFM),以处理立体声的特征特征特征对齐和遮挡问题。此外,由于空间信息对于超像素分割至关重要,因此我们进一步设计一个动态空间嵌入模块(DSEM)以重新添加空间信息,并且将通过DSEM中的DSEM进行自适应调整空间信息的权重(DF)用于实现更好的细分。全面的实验结果表明,我们的方法可以在KITTI2015和CityScapes数据集上实现最新性能,并且还可以在NJU2K数据集上的显着对象检测中验证效率。源代码将在接受纸张后公开提供。
translated by 谷歌翻译
在本文中,我们提出了一种基于排名的水下图像质量评估(UIQA)方法,该方法缩写为Uranker。乌兰克(Uranker)建立在高效的注意力图像变压器上。在水下图像方面,我们特别设计(1)直方图嵌入了水下图像作为直方图表的颜色分布以参加全局降解,以及(2)与模型局部降解的动态跨尺度对应关系。最终预测取决于不同量表的类代币,该标记是全面考虑多尺度依赖性的。随着保证金排名损失,我们的乌员可以根据其视觉质量通过不同的水下图像增强(UIE)算法来准确对同一场景的水下图像的顺序进行排名。为此,我们还贡献了一个数据集,即Urankerset,其中包含不同的UIE算法和相应的感知排名增强的足够结果,以训练我们的uranker。除了Uranker的良好表现外,我们发现一个简单的U-Shape UIE网络与我们的预训练的Uranker相结合时可以获得有希望的性能。此外,我们还提出了一个标准化尾巴,可以显着提高UIE网络的性能。广泛的实验证明了我们方法的最新性能。讨论了我们方法的关键设计。我们将发布我们的数据集和代码。
translated by 谷歌翻译
潜在空间基于能量的模型(EBM),也称为基于能量的先验,引起了对生成建模的日益兴趣。由于其在潜在空间的配方和强大的建模能力方面的灵活性所推动,最近构建的作品已经进行了有趣的尝试,目的是针对文本建模的解释性。但是,潜在空间EBM还继承了数据空间中EBM的一些缺陷。实践中退化的MCMC抽样质量会导致培训中的发电质量和不稳定差,尤其是在具有复杂潜在结构的数据上。受到最近的努力的启发,该努力利用扩散恢复的可能性学习是解决抽样问题的一种方法,我们在变异学习框架中引入了扩散模型和潜在空间EBM之间的新型共生,这是潜在扩散能量基于能量的模型。我们与信息瓶颈共同开发基于几何聚类的正则化,以进一步提高学到的潜在空间的质量。对几个具有挑战性的任务进行的实验证明了我们模型在可解释的文本建模上的优越性能而不是强大的同行。
translated by 谷歌翻译
从单个图像的面部图像动画取得了显着的进展。然而,当只有稀疏的地标作为驱动信号时,它仍然具有挑战性。鉴于源人面部图像和一系列稀疏面部地标,我们的目标是生成模仿地标运动的脸部的视频。我们开发了一种高效有效的方法,用于从稀疏地标到面部图像的运动转移。然后,我们将全局和局部运动估计结合在统一的模型中以忠实地传输运动。该模型可以学习从背景中分割移动前景并不仅产生全局运动,例如面部的旋转和翻译,还可以微妙地进行诸如凝视变化的局部运动。我们进一步改善了视频的面部地标检测。随着时间上更好地对齐的训练的标志性序列,我们的方法可以产生具有更高视觉质量的时间相干视频。实验表明,我们实现了与最先进的图像驱动方法相当的结果,在相同的身份测试和交叉标识测试上的更好结果。
translated by 谷歌翻译
了解网格单元如何执行路径集成计算仍然是一个根本的问题。在本文中,我们对网格单元进行了对路径集成的一般表示模型的理论分析,其中2D自身位被编码为更高的尺寸向量,并且通过向量的一般转换表示2D自动。我们确定转型的两个条件。一个是路径集成所必需的组表示条件。另一个是一种各向同性的缩放条件,可确保局部共形地嵌入,使得向量表示中的误差符合在2D自身位置中的误差。然后,我们调查最简单的转换,即线性变换,将其显式代数和几何结构揭示为矩阵旋转,并探索各向同性缩放条件与特殊类六角网格图案之间的连接。最后,通过基于优化的方法,我们可以学习六边形网格模式,该网格图案在啮齿动物大脑中共享网格细胞的相似性质。学习模型能够准确地长距离路径集成。代码可在https://github.com/ruiqigao/grid-cell-path中获得。
translated by 谷歌翻译
Strong lensing in galaxy clusters probes properties of dense cores of dark matter halos in mass, studies the distant universe at flux levels and spatial resolutions otherwise unavailable, and constrains cosmological models independently. The next-generation large scale sky imaging surveys are expected to discover thousands of cluster-scale strong lenses, which would lead to unprecedented opportunities for applying cluster-scale strong lenses to solve astrophysical and cosmological problems. However, the large dataset challenges astronomers to identify and extract strong lensing signals, particularly strongly lensed arcs, because of their complexity and variety. Hence, we propose a framework to detect cluster-scale strongly lensed arcs, which contains a transformer-based detection algorithm and an image simulation algorithm. We embed prior information of strongly lensed arcs at cluster-scale into the training data through simulation and then train the detection algorithm with simulated images. We use the trained transformer to detect strongly lensed arcs from simulated and real data. Results show that our approach could achieve 99.63 % accuracy rate, 90.32 % recall rate, 85.37 % precision rate and 0.23 % false positive rate in detection of strongly lensed arcs from simulated images and could detect almost all strongly lensed arcs in real observation images. Besides, with an interpretation method, we have shown that our method could identify important information embedded in simulated data. Next step, to test the reliability and usability of our approach, we will apply it to available observations (e.g., DESI Legacy Imaging Surveys) and simulated data of upcoming large-scale sky surveys, such as the Euclid and the CSST.
translated by 谷歌翻译
Classifier-free guided diffusion models have recently been shown to be highly effective at high-resolution image generation, and they have been widely used in large-scale diffusion frameworks including DALLE-2, Stable Diffusion and Imagen. However, a downside of classifier-free guided diffusion models is that they are computationally expensive at inference time since they require evaluating two diffusion models, a class-conditional model and an unconditional model, tens to hundreds of times. To deal with this limitation, we propose an approach to distilling classifier-free guided diffusion models into models that are fast to sample from: Given a pre-trained classifier-free guided model, we first learn a single model to match the output of the combined conditional and unconditional models, and then we progressively distill that model to a diffusion model that requires much fewer sampling steps. For standard diffusion models trained on the pixel-space, our approach is able to generate images visually comparable to that of the original model using as few as 4 sampling steps on ImageNet 64x64 and CIFAR-10, achieving FID/IS scores comparable to that of the original model while being up to 256 times faster to sample from. For diffusion models trained on the latent-space (e.g., Stable Diffusion), our approach is able to generate high-fidelity images using as few as 1 to 4 denoising steps, accelerating inference by at least 10-fold compared to existing methods on ImageNet 256x256 and LAION datasets. We further demonstrate the effectiveness of our approach on text-guided image editing and inpainting, where our distilled model is able to generate high-quality results using as few as 2-4 denoising steps.
translated by 谷歌翻译
本文考虑了使用嵌入式设备来获取和分类图像的设置。由于计算能力有限,嵌入式设备依赖于具有不平衡精度的简约分类模型。当认为本地分类不准确时,设备可以决定使用更准确但资源密集型的模型将图像卸载到边缘服务器。但是,资源限制(例如,网络带宽)需要调节这种传输,以避免交通拥堵和高延迟。当传输调节是通过令牌桶时,该论文调查了此卸载问题,该机制通常用于此类目的。目的是设计一种轻巧的在线卸载策略,该策略在令牌存储桶的限制下优化了特定于应用程序的指标(例如,分类精度)。该论文制定了基于深Q网络(DQN)的政策,并证明了其功效和在嵌入式设备上部署的可行性。值得注意的是,该策略可以处理复杂的输入模式,包括图像到达中的相关性和分类精度。评估是通过使用来自Imagenet图像分类基准生成的合成痕迹对局部测试床进行图像分类进行的。这项工作的实施可在https://github.com/qiujiaming315/edgeml-dqn上获得。
translated by 谷歌翻译